Stochastic Configuration Networks Research Articles

A novel stochastic configuration network with enhanced feature extraction for industrial process modeling

Stochastic configuration networks (SCNs), possessing sound generalization performance and low computational burden, have been extensively investigated in data analysis field. But in practice, it performs poor. This is mainly caused by two aspects: (1) the increment of random hidden nodes with incompact constraints may bring the emergence of redundant nodes, thereby resulting in poor modeling performance, and (2) SCNs have limited feature extraction capabilities. To address these issues, a novel SCN inserting feature layer, termed as FSCN, is presented in this paper. First, the embedded feature layer establishes a connection between the input and hidden layers, thus mapping the input data into a suitable feature space. Second, an inequality constraint based on Greville iterative method is established. It can not only facilitate the construction of high-quality hidden nodes but also guarantee convergence of the built model. Then, the Greville iterative method is also employed for output weight updating. Finally, comparative experiments on a real-valued nonlinear function, four real-world data sets and the modeling of two real industrial processes are conducted and the comparison results with other modeling methods show that FSCN has superior performance in terms of both model accuracy and compactness.

An information entropy-based fuzzy stochastic configuration network for robust data modeling

In data-driven modeling, the generalization performance of a neural network based on the minimum mean square error criterion may be affected when the samples contain noise. To handle uncertain data modeling problems, this study contributes a novel robust data modeling method called information entropy-based fuzzy stochastic configuration networks (IEFSCNs), which incorporates the maximum correntropy criterion (MCC) to reflect the squared error of the training sample set and uses the quantized minimum error entropy (QMEE) criterion to mine the distribution structure of the modeling data and suppress outlier data. Fuzzy systems are designed to generate nodes in the fuzzy transformation layer. A nonlinear enhancement layer is established as a hidden layer to enhance the representational capacity of the target system. The fuzzy transformation and nonlinear enhancement layers are fully connected to determine the outputs of the IEFSCNs. In addition, an incremental parameter learning algorithm is developed based on a supervision mechanism for optimizing the cost function, and its convergence property is demonstrated. Owing to the superiorities of the cost function by synthesizing MCC and QMEE, IEFSCNs can improve the modeling accuracy and reduce noise suppression. The performances are evaluated using various regression and classification tasks. Compared with existing methods, including SCNs, RVFL, F-SCNs, RSC-KDE, RSC-MCC, and SM-RSC, the proposed method demonstrates an effective robust modeling performance by reducing network structure redundancy while enhancing its ability to suppress noisy data.

An adaptive strategy for time‐varying batch process fault prediction based on stochastic configuration network

AbstractFault prediction ensures safe and stable production, and cuts maintenance costs. Due to the changing operating conditions that lead to the changes in the characteristics of industrial processes, there is a need to monitor the fault state of batch processes in real‐time and to accurately predict fault trends. An adaptive slow feature analysis‐neighborhood preserving embedding‐improved stochastic configuration network (SFA‐NPE‐ISCN) algorithm for batch process fault prediction is proposed. Firstly, SFA is used to extract the time‐varying features of process data and establish the update index of the NPE model. Then, to extract local nearest‐neighbor features and reconstruct them by the NPE model with adaptive update capability, square prediction error (SPE) statistics are constructed as fault state features based on the reconstructed error. Further, the hunter‐prey optimization (HPO) algorithm optimizes the weights and biases in the stochastic configuration network, and the singular value decomposition (SVD) and QR decomposition of column rotation are introduced to solve the ill‐posed problem of SCN and obtain the prediction model of ISCN. Finally, the obtained statistics SPE is formed into a time series, and the ISCN model is used to predict the process state trend. The effectiveness of the proposed algorithm is verified by case studies of industrial‐scale penicillin fermentation processes and the Hot strip mill process.

Improved stochastic configuration networks with vision patch fusion method for industrial image classification

This paper contributes to the advancement of stochastic configuration neural networks (SCN) in the field of visual applications. The proposed image classification randomized algorithm is an extension of deep stochastic configuration networks (DeepSCN), known as an improved SCN with vision patch fusion (Vi-SCN). Compared to existing two dimensional stochastic configuration networks (2DSCN) and DeepSCN, we have developed an incremental modeling method that employs a stochastic configuration patch fusion method. This method extracts randomly fused image features from three-channel high-resolution image data, improving the network's ability to extract features. Moreover, we have introduced a strategy for dynamically determining the depth structure of the network, enabling flexible adjustments to the network structure and the number of nodes in each layer based on the complexity of image recognition tasks. Through a series of comparisons on four image classification benchmark datasets, we assess the superior learning performance of our design compared to 2DSCN and DeepSCN. Furthermore, to evaluate the performance of Vi-SCN in practical visual application scenarios, we collect three industrial datasets with distinct characteristics. Vi-SCN demonstrate outstanding performance in all three tasks, showing its significant potential in image recognition.

A new two-stage decomposition and integrated hybrid model for short-term wind speed prediction

Accurate wind speed prediction is of essential importance for the stability and safe operation of power systems. Given the complexity of wind speed sequence, this paper proposed a new two-stage decomposition and integrated hybrid model to improve the accuracy of wind speed prediction. A two-stage decomposition method combining robust local mean decomposition (RLMD), sample entropy (SE) and variational modal decomposition (VMD) was used to decompose the wind speed signal in the data preprocessing stage. Firstly, the wind speed signal was decomposed into various components by RLMD, and the complexity of each component was calculated using the SE to classify them into random, detail component and trend component. Then, a secondary decomposition of the random component with the highest SE was performed using the VMD. In the prediction stage, two different prediction models were used for prediction depending on the smoothness of each component. Stochastic configuration networks (SCN) was used to predict the detail and trend components with relatively smoothness. Echo state network (ESN) was used to predict the components of the secondary decomposition. Finally, the actual wind speed data were compared by different prediction models, which illustrated that the prediction method proposed in this paper had good prediction accuracy and generalizability.

Universal approximation property of stochastic configuration networks for time series

For the purpose of processing sequential data, such as time series, and addressing the challenge of manually tuning the architecture of traditional recurrent neural networks (RNNs), this paper introduces a novel approach-the Recurrent Stochastic Configuration Network (RSCN). This network is constructed based on the random incremental algorithm of stochastic configuration networks. Leveraging the foundational structure of recurrent neural networks, our learning model commences with a modest-scale recurrent neural network featuring a single hidden layer and a solitary hidden node. Subsequently, the node parameters of the hidden layer undergo incremental augmentation through a random configuration process, with corresponding weights assigned structurally. This iterative expansion continues until the network satisfies predefined termination criteria. Noteworthy is the adaptability of this algorithm to handle time series data, exhibiting superior performance compared to traditional recurrent neural networks with similar architectures. The experimental results presented in this paper underscore the efficacy of the proposed RSCN for sequence data processing, showcasing its advantages over conventional recurrent neural networks in the context of the performed experiments.

Stochastic configuration networks with CPU-GPU implementation for large-scale data analytics

Stochastic configuration networks (SCNs) are a class of randomized learner models that have garnered increasing attention in data analytics. In the original implementation of SCN, the pseudo-inverse of the hidden layer output matrix is used to evaluate output weights. However, it is quite challenging to compute the pseudo-inverse of the hidden layer output matrices for large-scale datasets with high complexity, which occurs in many real-world applications. This paper aims to accelerate the learning process of SCNs by employing the well-known alternating direction method of multipliers (ADMM) solver and CPU-GPU heterogeneous computing technique. Empirical results on two benchmark datasets demonstrate the efficiency and effectiveness of the proposed method for large-scale data analytics.

Working condition recognition of fused magnesium furnace based on stochastic configuration networks and reinforcement learning

Automatic and accurate recognition of abnormal working conditions of fused magnesia furnace is of great significance to the safe and reliable production of fused magnesia. Aiming at the defects of manual judgment of abnormal working conditions in the production process of fused magnesium furnace and the existing recognition method of abnormal working conditions based on machine learning, this paper proposes a working condition recognition model for fused magnesium furnace based on stochastic configuration networks and reinforcement learning. Firstly, a hybrid data augmentation method of generative and non-generative is used to obtain high-quality sample data with salient features. Secondly, based on ResNeXt, multi-scale local features are extracted under the condition of limited parameter quantity by controlling the cardinality. Combining a mixed model that enjoys the benefit of both self-attention and convolution (ACmix) and bidirectional feature pyramid network (BiFPN), the extracted local feature maps of different scales are cross-scale fused and focused, and more differentiated detailed feature information of the region of interest is retained. Thirdly, based on Transformer, the working condition recognition network of fused magnesium furnace is constructed to improve the global correlation between adjacent local features in the spatial dimension. The fused features are sent to stochastic configuration networks to establish a classification criterion for working condition recognition of fused magnesium furnace with generalization ability. Finally, reinforcement learning is used to evaluate the credibility of uncertain recognition results of samples in real time, and a self-optimizing adjustment action strategy at the Transformer encoding layer is defined. Build a library of Transformer models with different encoding layers, which is adapt to the different feature extraction requirements of multi-modal working samples. The experimental results show that the method in this paper has better recognition performance and generalization ability than other algorithms.

Online Self-Learning Stochastic Configuration Networks for Nonstationary Data Stream Analysis

Online Self-Learning Stochastic Configuration Networks for Nonstationary Data Stream Analysis

A cooperative stochastic configuration network based on differential evolutionary sparrow search algorithm for prediction

Stochastic configuration network (SCN) is a powerful prediction model whose performance is significantly influenced by the configuration of the network parameters. To improve the prediction accuracy of the network, a cooperative stochastic configuration network (CSCN) based on a novel differential evolutionary sparrow search algorithm (DESSA), termed as DESSA-CSCN, is proposed. In the CSCN, the number of hidden layer nodes is adaptively adjusted according to the number of iterations, and the parameters of hidden nodes are cooperatively optimized by using a population-based metaheuristic algorithm. A sparse matrix is introduced to mitigate parameter overfitting caused by the increased number of hidden layer nodes. During parameter optimization, the fitness function is constructed by using the supervision mechanism of the SCN, and the DESSA is utilized as the metaheuristic algorithm to update the weights and biases. In order to verify the effectiveness of the DESSA-CSCN, several simulation experiments have been conducted. The performance of the DESSA is evaluated by the CEC2017 test suit, and the simulation results show that the DESSA exhibits better convergence accuracy and can jump out of local optima more effectively than other algorithms. The performance of the DESSA-CSCN is evaluated by 4 datasets from KEEL, and the simulation results indicate that the DESSA-CSCN achieves better prediction accuracy and faster prediction speed than other models.

Multi-target regression via stochastic configuration networks with modular stacked structure

Multi-target regression via stochastic configuration networks with modular stacked structure

Multi-task stochastic configuration network with autonomous linking and its application in wastewater treatment processes

Stochastic configuration networks (SCNs) have been widely used for modeling complex industrial process due to their rapid learning speed, ease of implementation, and universal approximation capability. For modeling water quality parameters in wastewater treatment processes (WWTP), however, multiple complex tasks are often required to be modelled simultaneously. In this paper, a multi-task stochastic configuration network with autonomous linking characteristic is proposed to further develop the modeling capability of SCNs to deal with multi-tasks and achieve simultaneous measurement of multiple critical water quality parameters in the WWTP. The method can autonomously construct corresponding common nodes and proprietary nodes according to the distribution characteristics of different tasks to model the shared and private information among these tasks. Specifically, the relevant information between these tasks is explored by constructing common nodes; then personalized approximation of each task is achieved by constructing proprietary nodes for different tasks, thus improving the overall modeling performance of the model. A series of benchmark experiments and an industrial case from WWTP are carried out to verify the superiority of the proposed method. Experimental results demonstrate that our proposed method has a promising potential for multi-task data modeling.

A novel method of rolling bearings fault diagnosis based on singular spectrum decomposition and optimized stochastic configuration network

Fault diagnosis of rolling bearings is essential for the safe operation of rotating machinery. However, in the production process, the rolling bearings have a complex working environment embedded with weak fault signals and a large number of interfering signals, which is a considerable challenge to automatically and accurately detect bearing fault type from the actual vibration signal. Therefore, a novel fault diagnosis scheme is proposed based on singular spectrum decomposition (SSD) and an optimized stochastic configuration network (SCN). Firstly, SSD is used to pre-process the original rolling bearings vibration signal to obtain several singular spectral components (SSCs) and the practical component is selected according to the maximum correlation coefficient for signal reconstruction. Furthermore, time domain and power spectrum entropy (PSE) features of the reconstructed signal are extracted to obtain a fault information-rich feature sets. In addition, the parameters of the SCN are optimized by marine predators algorithm (MPA) to enhance the learning ability and generalization performance of the SCN. Finally, the feature sets are input into MPA-SCN to achieve fault classification. Experimental results exhibit that the proposed method has higher accuracy in rolling bearings fault diagnosis compared with other methods, which provides a high-efficiency solution for rolling bearings fault diagnosis.

Video-based working condition recognition of fused magnesium furnace with stochastic configuration networks

Working condition recognition of a fused magnesium furnace (FMF) suffers from unbalanced under-burning condition samples, inconsistent quality of training samples and difficulty in characterizing dynamic production processes with images. This paper presents a novel approach to detect the under-burning working condition in FMF based on a 3D-cycle-generative adversarial network (3D-cycle-GAN) and Video Swin Transformer-Stochastic Configuration Networks (Video Swin-SCNs). Firstly, to resolve the temporal discontinuity defect caused by Cycle-GAN through the visual appearance of video composite frames, we construct a motion consistency-based 3D-Cycle-GAN model that considers the visual appearance and temporal continuity constraints of unpaired video transitions and is designed to generate video samples of under-burning working conditions. Secondly, a reinforcement learning approach is used to assess the value of the video quality and to filter out possible low-quality samples generated. Finally, the local attention is extended from the spatial domain to the spatial-temporal domain to solve the difficulty in characterizing the dynamic production process with the static images. The spatial-temporal features from the Video Swin Transformer are fed into SCNs to classify the working conditions. The experiment results indicate the effectiveness and feasibility of the proposed method.

A wind speed interval prediction method for reducing noise uncertainty

Due to the noise uncertainty, the conventional point prediction model is difficult to describe the actual characteristics of wind speed and lacks a description of the wind speed fluctuation range. In this paper, the kernel density estimation according to its error value is given, and then its fluctuation range is found to combine the prediction results of the test set to get its prediction range. Firstly, the singular spectrum analysis (SSA) is introduced to conduct the noise reduction, and variational modal decomposition (VMD) is performed to handle the sequences, then an improved slime mold algorithm (SMA) is proposed to optimize the VMD, and the stochastic configuration networks (SCNs) is applied to perform the prediction. Finally, the interval prediction results are calculated by fusing the point prediction error and kernel density estimation. The experimental results demonstrate that the proposed method can effectively reduce the noise interference in the wind speed prediction.

Prediction of X-ray fluorescence copper grade using regularized stochastic configuration networks

In copper ore flotation processes, accurate measurement and estimation of the copper grade of the final products are crucial for field operations and control. Due to the presence of multicollinearity and outliers in the measured data, the existing on-stream X-ray fluorescence grade analyzer is inaccurate. To address these issues, this study proposes a robust data modeling algorithm to improve the performance of copper grade prediction. Specifically, a regularized stochastic configuration network and a generalized M−estimation method are used to overcome the uncertainties in the processing data. The experimental results clearly demonstrate that the proposed model outperforms other modeling algorithms in terms of prediction accuracy and robustness.

Fishing risky behavior recognition based on adaptive transformer, reinforcement learning and stochastic configuration networks

The fishing behavior in a high-voltage environment may lead to electric shock accidents. This paper proposes a fishing risky behavior recognition method based on adaptive Transformer, reinforcement learning and stochastic configuration networks (SCNs). Firstly, the fishing behavior image sample training set is evaluated using the actor network based on linear entropy in the reinforcement learning module to select suitable training samples that possess abundant straight-line elements. Subsequently, the selected training samples are fed into a multi-scale spatial deep convolution to extract continuous spatial features of elongated objects. The deformable Transformer network, enhanced with adaptive encoding and decoding layers through SCNs, is used to obtain the position and detection results of the fishing rod. The trained Transformer model is evaluated by a defined credibility evaluation metric for acquiring rewards to update the training sample set iteratively. Then, an adaptive adjustment mechanism is constructed for the encoding and decoding layers of the adaptive Transformer network to establish a library of adaptive Transformer models with different encoding and decoding layer levels to accommodate the feature requirements of training samples in various scenarios. Finally, the blending learning method for combining the detection results from the model library is integrated with human body object detection and pose estimation methods, and a predefined expert system is utilized for logic reasoning on fishing risky behavior. Experimental results demonstrate the effectiveness of the proposed method in this paper.

Short-term household load forecasting based on Stacking-SCN

The household load has high volatility and uncertainty, which plays an increasingly important role in short-term power load forecasting for individual residential customers in the operation and planning of future power grids. There is still room for improvement in the prediction accuracy of traditional machine learning and deep learning models. This paper firstly analyzes the complexity of household load data and formulates forecasting problems. Then, the principle of stochastic configuration network (SCN), stacking algorithm and a stacking integrated stochastic configuration network (Stacking-SCN) is introduced, which constructs a short-term household load forecasting model. By adaptively constraining the output weight distribution of the network model, the prediction accuracy can be improved. To show the prediction performance of the algorithm, the experiment comparing the Stacking-SCN model with incremental random vector functional link network (IRVFLN) models, the traditional SCN, ensemble-SCN (E-SCN), deep neural networks (DNN) and long short-term memory (LSTM), and verify it with RMSE, MAE and APE performance metrics. Experimental results show that Stacking-SCN can more effectively express the time series relationship of household short-term load, has higher prediction accuracy, effectively prevents network over-fitting problems, and improves the accuracy of prediction capabilities.

Intuitionistic Fuzzy Stochastic Configuration Networks for Solving Binary Classification Problems

Intuitionistic Fuzzy Stochastic Configuration Networks for Solving Binary Classification Problems

Decentralized Semi-Supervised Learning for Stochastic Configuration Networks Based on the Mean Teacher Method

Decentralized Semi-Supervised Learning for Stochastic Configuration Networks Based on the Mean Teacher Method